Skip to content

ci: scope OceanBase acceptance independently from harness replay - #2526

Open
Zxf-xufeng wants to merge 30 commits into
masterfrom
codex/scope-oceanbase-ci
Open

Zxf-xufeng wants to merge 30 commits into
masterfrom
codex/scope-oceanbase-ci

Conversation

@Zxf-xufeng

@Zxf-xufeng Zxf-xufeng commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

Which issue or RFC does this PR close?

Depends on #2515, which should merge first. This branch includes its reliability fixes so CI validates their combined behavior. No issue is automatically closed.

Rationale for this change

Replay changes unnecessarily trigger native OceanBase regressions, while native-only changes start both replay workers. Several SQLite-only tests also trigger the broader database suite. Select these workloads independently while requiring complete evidence for every selected workload.

What changes are included in this PR?

  • Separate replay (harness) from native OceanBase regressions (oceanbase_acceptance), with explicit ownership for shared helpers and Compose inputs.
  • Select SQLite and OceanBase for replay, one OceanBase worker for native-only acceptance, and a shared OceanBase worker/container when both are selected.
  • Run bounded startup independently of native-test selection. Validate and upload native JUnit separately; require successful selected steps and evidence, including replay-only runs.
  • Keep the combined OceanBase budget at 90 minutes: 15 startup + 50 native acceptance + 2 diagnostics + 15 replay, leaving 8 minutes for remaining evidence and cleanup.
  • Exempt six audited SQLite/in-memory files from direct database-suite matching; retain normal Python coverage and all other matching scopes. Shared inputs and full fallbacks still take precedence.
  • Reuse Main's lint/format checks while retaining harness-specific validation; standalone harness checks remain complete.
  • Include ci(oceanbase): bound startup and balance migration workers #2515's exactly-once E2E execution, duration-balanced migration workers, bounded native diagnostics, cache headroom, schema reuse and CI/bootstrap fixture repairs. Preserve master's additional legacy/native FTS coverage.
  • Download pinned GitHub Release inputs through a bounded official API fallback when the release-download route fails. Both paths retain the locked SHA256 and share one total deadline. The same helper prepares pinned skill-up and Windows NSIS tools before their consumers run.

Are there any user-facing changes?

No public API, persistence or release-compatibility changes. This PR changes CI selection and integration with #2515.

How was this change tested?

  • make check: passed after merging master 7fef43d6.
  • Managed-resource acquisition and resource-store tests: 62 passed, 6 native-platform skips. The fallback tests retain digest rejection and the overall download budget; a local live archive download hit the local network deadline, so native download validation remains with CI.
  • Master subscription/SDK integration regressions: 516 passed; SQLite authorization: 20 passed.
  • SQL scoring and native FTS capability tests: 40 passed. The scoring fixture implements the synchronous connection adapter required by capability probing, retaining all score/order assertions.
  • Shared download/resource, CI path selection and E2E workflow regressions: 268 passed, 8 platform-dependent skips.
  • The shared fixes also passed SQLite authorization scenarios, 8 DSH real-server E2E tests, 10 OpenDAL connector tests and 2 ACP bootstrap tests.
  • Workflow action pins and diff checks passed. A Git merge-tree check confirms the two PR heads combine without conflicts.
  • Full CI on the pushed head is pending. The 82/90-minute budget is locally verified; a complete native acceptance pass remains required. Local Rust dependency downloads timed out, so desktop tests require CI validation.

AI usage statement

OpenAI Codex (GPT-6) assisted with analysis, implementation, regression tests and PR preparation.

Comment thread .github/workflows/e2e-harness.yml Outdated
if: ${{ !cancelled() && needs.validate.result == 'success' }}
runs-on: ${{ (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) && 'powercontext-ci-standard' || 'ubuntu-24.04' }}
# Reserve 50m for OceanBase regressions, 15m for replay, and 10m for setup/evidence/cleanup.
# Keep the existing overall budget; selected suites share setup and cleanup.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 验收作业的分步预算之和 80 分钟超过作业上限 75 分钟,而最常见的改动就会走到这条路径

把 OceanBase 启动拆成独立步骤后,本作业三步上限是 15(oceanbase_setup)+ 50(native_acceptance)+ 15(acceptance_scenarios)= 80 分钟,而 timeout-minutes 仍是 75,本行注释又写"保持既有总预算"。base 里数据库启动在同一个 50 分钟步骤内执行,合计 50 + 15 = 65,还给证据扫描和 teardown 留了 10 分钟。

影响:oceanbase_acceptance 已加入 [shared.runtime].suites,任何 src/**、openapi/**、pyproject.toml、uv.lock 改动都会同时选中两个范围,也就是最常见的一类 PR。当 OceanBase 起得慢(up --wait-timeout 660 已允许 11 分钟,参数轮询最多再 60 秒)、原生用例又接近 50 分钟上限时,作业会在 75 分钟处被整体取消,Require all selected acceptance work and evidence 来不及执行,失败只表现为 job 超时,看不出是哪一段验收没跑。

建议:让三步之和不超作业上限(例如 native_acceptance 降为 45 分钟,或上限提到 85 并同步改注释),并把 teardown 与证据上传的时间也计入;tests/test_e2e_workflow_contract.py 可以顺手断言"分步上限之和 ≤ 作业上限",避免以后再漂。

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 342cf9ad. The OceanBase job now allows 90 minutes: 15 for startup + 50 for native acceptance + 15 for replay, with 10 minutes reserved for diagnostics, evidence and cleanup. A regression check covers combined, native-only and replay-only budgets.

I also integrated #2515 at 08a34940 into this branch and fixed the combined workflow: the container ID is passed through a setup output, the native deadline starts in its own step, and JUnit validation/upload use the same file. Changes to the shared OceanBase helper now select native acceptance as well. Pre-teardown diagnostics are preserved.

The regression checks fail against the uncorrected combination and pass with the fix; local checks and workflow lint pass. Real OceanBase acceptance still requires the new CI run. This PR now depends on #2515, which should be merged first; the #2515 branch was not changed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[已修复] 分步预算改为 15(setup) + 50(native) + 2(diagnostics) + 15(replay) = 82 ≤ 作业上限 90(原 15+50+15=80 对 75)。新增的 oceanbase_setup 步骤内部也自洽:timeout 700s compose up --wait 加 oceanbase_ci.py wait --timeout 90 共 790s < 步骤上限 900s;native 步骤 native_deadline=$((SECONDS + 2850))(47.5 min)< 步骤上限 50 min。

补充证据(exact head 105e7a94,WSL venv 3.13.11):tests/test_oceanbase_ci.py + tests/test_ci_changes.py + tests/test_e2e_workflow_contract.py 共 178 passed。

验证缺口(不构成新 Finding,但影响合并判断):该 head 的 Main run 42390978248 结论是 cancelled,其中 tests (3.12, e2e) 被取消;suite 作业的 if 含 contains(fromJSON('["success","skipped"]'), needs.tests.result),于是 CI preflight 明明输出全 true(含 harness / oceanbase_acceptance / upgrade),仍有 30 个 suite 作业 skipped,Acceptance (oceanbase) 连 check 都没有产生。因此 82/90 的预算与本 PR 新增的 setup 路径目前没有一次完整绿跑作为证据。

另核对 scripts/oceanbase_ci.py 的解析列位:values = {row[4]: row[6] ...} 与 e2e-harness.yml 中 awk '$5 == "datafile_size" && $7 == "4G"' 一致,对应 SHOW PARAMETERS 的 name / value 列(4.x 输出为 zone, svr_type, svr_ip, svr_port, name, data_type, value, …)。测试 fixture 的 7 列形状与之一致,不是测试迎合实现造成的错位。

@Zxf-xufeng Zxf-xufeng Nov 9, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for confirming the corrected 82/90-minute budget. The latest master conflict resolution and CI fixes are pushed in b95e6a5. Both branches include master 7fef43d, pass make check, and combine locally without conflicts. The scope-specific workflow selection remains on this branch.

Native authorization scenarios reuse only the module-scoped database schema, while retaining separate Scopes, idempotency keys and Server lifecycles. Migration-recovery scenarios keep isolated databases. The resource producer also has a checksum-preserving official Release API fallback for the observed HTTP 504 download failures.

The previous run passed Python 3.11–3.13, but its 3.14 unit runner lost communication with GitHub. A failed-only rerun passed that unit job; remaining validation was superseded by this conflict-resolution push. These results do not close the native validation gap. Run 42573616986 validates the newly pushed head; I will report the actual backend outcome before claiming acceptance is complete.

The same bounded, checksum-verified downloader now covers skill-up and pre-staged Windows NSIS tools. The plugin’s ordinary release URL reproduced HTTP 504 locally, while the official asset API returned bytes matching both the pinned SHA256 and Tauri’s upstream SHA1. This verifies the alternate artifact route; the native Windows installer and backend suites still require the linked CI run.

@hireview001 hireview001 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • CI failed: CI result (failure).
  • CI failed: Upgrade release gate (failure).
  • CI failed: CI preflight (failure).
  • CI failed: Main (failure). Failed steps: CI preflight / Check License Header; Upgrade release gate / Require the selected parallel acceptance workflow; CI result / Require successful selected checks.

@knqiufan knqiufan added the 1.5.1 v1.5.1 release tracking label Nov 7, 2026

@knqiufan knqiufan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing acceptance-timeout finding still applies: the combined OceanBase setup, native and replay steps can consume more than the enclosing job budget, leaving no guaranteed time for evidence and cleanup. Please address the existing thread. The focused scope and workflow tests passed locally; no real OceanBase job was rerun.

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing P2 acceptance-timeout finding remains valid at this head. Extracting setup increases the combined step budget from 65 to 80 minutes while the OceanBase job still allows only 75 minutes, so a slow combined run can be cancelled before evidence collection and cleanup. Please make the total fit within the job deadline with time reserved for those final steps. No additional independent findings.

@Zxf-xufeng

Copy link
Copy Markdown
Member Author

@knqiufan, in response to your review: fixed in 342cf9ad. The OceanBase job now allows 90 minutes, covering 15 minutes of startup, 50 minutes of native acceptance and 15 minutes of replay, with 10 minutes reserved for diagnostics, evidence and cleanup. A regression check protects that reserve for all three selection modes. I also replied in the original inline thread. Local checks pass; the new CI run is still in progress, so real OceanBase acceptance is not yet confirmed.

@Zxf-xufeng

Copy link
Copy Markdown
Member Author

@hidb4ai, in response to your review: addressed in 342cf9ad. The combined workload's 80-minute step budget now fits within a 90-minute job deadline, leaving 10 minutes for diagnostics, evidence and cleanup. The workflow comment and regression checks reflect this allocation. The branch also includes #2515's fixes and repairs the cross-step container/deadline handoff and JUnit upload path. Local validation passes; the new CI run is still in progress.

@Zxf-xufeng

Copy link
Copy Markdown
Member Author

@hireview001, following up on your CI failure review after the new run 42380905358 completed for 342cf9ad: it also failed, but at a different point in license-tool setup.

The previous run failed when installing license-eye into /usr/local/bin without write permission. This branch includes master's writable-prefix change, and the new logs confirm INSTALL_DIR points to the runner's temporary directory. However, CI preflight hit the license step's 10-minute timeout: Go setup took about 6 minutes 51 seconds, followed by dependency downloads/building license-eye. The actual license scan was skipped, so successful installation and scanning remain unverified.

The upgrade gate records SCOPE_RESULT=failure and UPGRADE_RESULT=skipped; its failure and the final CI result are downstream consequences of preflight, not executed upgrade-test failures. Native OceanBase and upgrade acceptance did not run, so this run does not validate the acceptance fixes or all 12 upgrade configurations.

The CI issue is not yet verified as resolved. License-tool setup must complete successfully before the remaining acceptance can be validated.

@knqiufan knqiufan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@AlexStocks

Copy link
Copy Markdown
Contributor

@Zxf-xufeng plz fix the ci failure.

Zxf-xufeng and others added 2 commits November 8, 2026 21:12
Reuse the entrypoint fix from #2537 to unblock backend acceptance.

Co-authored-by: knqiufan <knqiufan@foxmail.com>
@hidb4ai hidb4ai removed the 1.5.1 v1.5.1 release tracking label Nov 9, 2026
@AlexStocks AlexStocks added the 1.6.0 v1.6.0 release tracking label Nov 9, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.6.0 v1.6.0 release tracking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants